Task alerts #721
Task alerts #721
Conversation
|
Alert if a task is killed due to slave decommissioning.
|
| getAlerts: (requestTaskHistory) => | ||
| task = @models.task | ||
| alerts = [] | ||
| # Is this a scheduled task that has been running much longer than previous ones? |
There was a problem hiding this comment.
instead of calculating this yourself, you should use averageRuntimeMillis in the SingularityDeployStatistics object, which can be grabbed from the deploy history endpoint (.../api/history/request/REQUEST_ID/deploy/DEPLOY_ID). it would also be wise to surface warnIfScheduledJobIsRunningPastNextRunPct from https://github.com/HubSpot/Singularity/blob/master/SingularityService/src/main/java/com/hubspot/singularity/config/SingularityConfiguration.java#L172 into the window.config object so that we don't have to hardcode the threshold value
|
Updated |
| @@ -63,6 +64,8 @@ public String getFinishedTaskLogPath() { | |||
| private boolean hideNewDeployButton = false; | |||
| private boolean hideNewRequestButton = false; | |||
|
|
|||
| private Optional<Integer> warnIfScheduledJobIsRunningPastNextRunPct; | |||
There was a problem hiding this comment.
i don't think this is necessary -- using the value inside SingularityConfiguration is probably good enough
| deployInfo = new DeployDetails | ||
| deployId: deployId | ||
| requestId: requestId | ||
| deployInfo.fetch().success => |
There was a problem hiding this comment.
technically we could only make this call only if it's needed (i.e. if the task is scheduled + currently running)
Show an alert on the task page if it is a scheduled task that has been running for longer than twice the average for tasks in the request. /cc @tpetr